Papers by Amir Hossein Kargaran
MaskLID: Code-Switching Language Identification through Iterative Masking (2024.acl-short)
Copied to clipboard
| Challenge: | Sentence-level LIDs are classifiers trained on monolingual texts to provide single labels, typically using a softmax layer to turn scores into probabilities. |
| Approach: | They propose a simple yet effective code-switching language identification method that uses the LID itself to mask features associated with L1 and L2 in the next round. |
| Outcome: | The proposed method is based on two open-source LIDs based in the FastText architecture and does not require any external resources. |
How Programming Concepts and Neurons Are Shared in Code Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Several studies have focused on programming languages in a monolingual setting, but most focus on programming language models. |
| Approach: | They perform a few-shot translation task on 21 PL pairs using two Llama-based models and decode the embeddings of intermediate layers. |
| Outcome: | The proposed model assigns high probability to English tokens in the second half of the intermediate layers and language-specific neurons are concentrated in the bottom layers . the model's concept space is closer to English (including PL keywords) and the model is more efficient at identifying language-related neurons. |
How Transliterations Improve Crosslingual Alignment (2025.coling-main)
Copied to clipboard
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Ayyoob ImaniGooghari, Orgest Xhelili, Haotian Ye, Chunlan Ma, François Yvon, Hinrich Schütze
| Challenge: | Recent studies show that post-aligning multilingual pretrained language models improve crosslingual alignment, but it is unclear how and why this is achieved. |
| Approach: | They propose to explicitly evaluate crosslingual alignment by adding transliterations to models using original and transliterated data. |
| Outcome: | The proposed approach improves crosslingual alignment even for random sentences. |
GlotScript: A Resource and Tool for Low Resource Writing System Identification (2024.lrec-main)
Copied to clipboard
| Challenge: | GlotScript is an open resource and tool for low resource writing system identification. |
| Approach: | They propose to use GlotScript to automatically identify writing systems for low resource languages . they demonstrate that Glotscript can help cleaning multilingual corpora . |
| Outcome: | The proposed tool can help clean multilingual corpora and provide insights on coverage of low resource scripts and languages by each language model. |
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment (2025.findings-acl)
Copied to clipboard
Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, François Yvon, Hinrich Schuetze
| Challenge: | Existing benchmarks for multilinguality for English-centric large language models focus on classic tasks or cover a minimal number of languages. |
| Approach: | They propose a method to assess multilingual capabilities of pre-trained LLMs using parallel sentences. |
| Outcome: | The proposed method evaluates the multilingual capabilities of pre-trained English-centric models using parallel sentences. |
On Relation-Specific Neurons in Large Language Models (2025.emnlp-main)
Copied to clipboard
Yihong Liu, Runsheng Chen, Lea Hirlimann, Ahmad Dawar Hakimi, Mingyang Wang, Amir Hossein Kargaran, Sascha Rothe, François Yvon, Hinrich Schuetze
| Challenge: | In large language models, certain neurons can store distinct pieces of knowledge learned during pretraining. |
| Approach: | They hypothesize that relation-specific neurons detect relation in input text and guide generation involving such a relation. |
| Outcome: | The proposed model can handle facts involving relation r and facts containing a different relation . |
CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely adopted for a diverse range of tasks, from highly skilled professionals to non-technical individuals. |
| Approach: | They propose a suite of lightweight adversarial attacks that allow LLMs to reveal harmful behavior during conversations. |
| Outcome: | The proposed model can recover from fabricated bias claim and reject biased follow-up questions. |
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages (2023.acl-long)
Copied to clipboard
Ayyoob ImaniGooghari, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, Hinrich Schütze
| Challenge: | Lack of LLMs supporting low-resource languages is a serious impediment to bringing NLP to all of the world. |
| Approach: | They create a model that scales LLMs horizontally and a corpus that covers 511 low-resource languages. |
| Outcome: | The proposed model improves on five diverse tasks across low- and high-resource languages. |
Tracing Multilingual Factual Knowledge Acquisition in Pretraining (2025.findings-emnlp)
Copied to clipboard
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Felicia Körner, Ercong Nie, Barbara Plank, François Yvon, Hinrich Schuetze
| Challenge: | Large Language Models are capable of recalling multilingual factual knowledge, but most studies evaluate only the final model, leaving the development of factual recall and crosslingual consistency unexplored. |
| Approach: | They trace how factual recall and crosslingual consistency evolve during pretraining, focusing on OLMo-7B as a case study. |
| Outcome: | The results show that fact frequency is the key to a better recall of multilingual facts, regardless of language, and some low-frequency facts in non-English languages can still be correctly recalled. |
Hengam: An Adversarially Trained Transformer for Persian Temporal Tagging (2022.aacl-main)
Copied to clipboard
| Challenge: | A wide array of natural language processing (NLP) applications relies on accurately identifying events and their respective occurrence times. |
| Approach: | They propose an adversarially trained transformer for Persian temporal tagging that can generalize over the HengamTagger’s rules. |
| Outcome: | The proposed tool outperforms state-of-the-art methods on a diverse and manually created dataset. |